The best choice problem for random number of objects with a refusal probability

In this note, wewill consider the best choice problem for random number of objects with a refusal probability. There aremany papers and books on this subject fromChow/Robbibs/Siegmund(1971), Siryaev(1973) downward. These problems has been close relations with the martingale theory, the potential theory, the optimality principle in the mathematical programming. And now it is derived the optimal rule by using the sum-the-odds theorem. Refer to Ferguson(2006), Bruss(2003), Tamaki(2001), Hsiau/Yang(2000) etc. It would be prefered to call it the odds-theorem rather than the secretary problem or the best choice nowdays. Here we adapt a classical analysis to obtain its optimal value for the problem on Markov processes. Firstly the formulation of stopping problem forMarkov processes are given and its optimal equation of the value are designated. Since the secretary problem has a special transition probability, we show the optimal value is calculated as One-Look ahead policy or monotone stopping problem. Then it is treated asymptotically and solve the value in the explicitly form by a differential equation. Some equivalence relation is shown between the decision-theoretic form of the optimal equation and the transition probability with refusal. In the last, by a translation from the case of random number of objects to the standard form of optimal stopping problem, we could obtain the optimal value in the cited case of title. Also a variation for the selection is considered. 1 Formulation and Notations The optimal stopping problem is a special case of Markov decision processes and the decision maker can either select to stop so as receiveing rewards, or to pay costs and continue observing the state. Let xn, n = 0, 1, 2, · · · be a Markov Chain with a transition probability P = P(i, j), i, j ∈ S over a state space S in R1. We assume that S is countable, but this is inessential and for our discussion so that the scaling limit case of problems with the random numbers can be treated. For a stopping time τ, the aim is to maximize the expectation of payoff v(i; τ) starting at i and its optimal value v(i) defined by v(i) = sup τ<∞ v(i, τ) = sup τ<∞ E [ r(xτ − τ ∑ n=0 c(xn) ∣∣∣ x0 = i ] , i ∈ S (1.1)